refactor!: drop reasoning trace extraction logic #1427
Conversation
…al_kwargs BREAKING CHANGE: Remove reasoning_config and apply_to_reasoning_traces config options - Remove ReasoningModelConfig, ParsedTaskOutput, and related extraction logic - Replace with automatic extraction from response.additional_kwargs['reasoning_content'] - Remove 1,085 lines of test code and related infrastructure - Simplify parse_task_output() to return str directly - Update DeepSeek-R1 and Nemotron example configs Reasoning traces are now automatically extracted by LangChain and stored in additional_kwargs, eliminating the need for token-based parsing and complex configuration options.
Pouyanpi
force-pushed
the
refactor/reasoning-traces-langchain-integration
branch
from
c4d3366 to
8aee801
Compare
Pouyanpi
changed the title
Codecov Report❌ Patch coverage is
📢 Thoughts on this report? Let us know! |
|
@Pouyanpi , shouldn't we keep |
Not anymore, now if someone wants to use a specific output rails that checks the reasoning content should include it in the prompt template (see https://github.com/NVIDIA-NeMo/Guardrails/pull/1432/files#diff-1eb0f307d462ecfbe7710b361f4d296d3648a44bfcf85fda895c6942b2e4606dR37-R39 ) and use a rail that supports guardrailing reasoning contents ( for example see how actions.py is modified) There needs to be a subsequent PR that updates some of the actions. |
I don't have anything against it, but for me it seemed like a useful option to be able to run any output rails on response only vs (thinking + response) by only adding a config flag. I agree that with the current implementation even more complex output rails can be defined way easier, but maybe it's not a bad idea to have the flag anyway. |
I understand your point. As of introducing these PRs we are treating bot thinking and thus reasoning traces as a distinct entities. I'll open a PR that addresses this, we can keep it for sake of backward compatibility until we have proper rails in place. |
…al_kwargs (#1427) BREAKING CHANGE: Remove reasoning_config and apply_to_reasoning_traces config options - Remove ReasoningModelConfig, ParsedTaskOutput, and related extraction logic - Replace with automatic extraction from response.additional_kwargs['reasoning_content'] - Remove 1,085 lines of test code and related infrastructure - Simplify parse_task_output() to return str directly - Update DeepSeek-R1 and Nemotron example configs Reasoning traces are now automatically extracted by LangChain and stored in additional_kwargs, eliminating the need for token-based parsing and complex configuration options (this is valid for nvidia_ai_endpoints and deepseek engines).
…al_kwargs (#1427) BREAKING CHANGE: Remove reasoning_config and apply_to_reasoning_traces config options - Remove ReasoningModelConfig, ParsedTaskOutput, and related extraction logic - Replace with automatic extraction from response.additional_kwargs['reasoning_content'] - Remove 1,085 lines of test code and related infrastructure - Simplify parse_task_output() to return str directly - Update DeepSeek-R1 and Nemotron example configs Reasoning traces are now automatically extracted by LangChain and stored in additional_kwargs, eliminating the need for token-based parsing and complex configuration options (this is valid for nvidia_ai_endpoints and deepseek engines).
…al_kwargs (#1427) BREAKING CHANGE: Remove reasoning_config and apply_to_reasoning_traces config options - Remove ReasoningModelConfig, ParsedTaskOutput, and related extraction logic - Replace with automatic extraction from response.additional_kwargs['reasoning_content'] - Remove 1,085 lines of test code and related infrastructure - Simplify parse_task_output() to return str directly - Update DeepSeek-R1 and Nemotron example configs Reasoning traces are now automatically extracted by LangChain and stored in additional_kwargs, eliminating the need for token-based parsing and complex configuration options (this is valid for nvidia_ai_endpoints and deepseek engines).
4 participants
Stack Info
This PR is part of a stack:
Description
Replaces custom reasoning trace extraction with LangChain's built-in
additional_kwargsapproach, removing ~1,500 lines of code.Breaking Changes
reasoning_config(includingstart_token,end_token,remove_reasoning_traces) removed from model configurationapply_to_reasoning_tracesremoved from output rails configurationMigration
Remove
reasoning_configsections from config files. Reasoning traces are now automatically extracted by LangChain for supported models (e.g., DeepSeek-R1).Changes
ReasoningModelConfig,ParsedTaskOutput, token extraction logicresponse.additional_kwargs['reasoning_content']